Dynamic neural networks, comparing spiking circuits and LSTM
نویسندگان
چکیده
We have investigated two specific network types in the class of dynamic neural networks: LSTM and spiking neural networks. Dynamic neural networks in general are computationally powerful and very promising for tasks in which temporal information has to be processed. We’d like to remark that this is the case for virtually any task or application interacting with the real world. We have tested the networks on a broad set of dynamic tasks and most problems were solved by both; there are some fields though where either LSTM or the spiking neural networks performed better. These differences can be largely brought back to the differences between second and third generation networks.
منابع مشابه
Exploring Temporal Memory of LSTM and Spiking Circuits
We have investigated two specific network types in the class of dynamic neural networks: LSTM and spiking neural networks. Dynamic neural networks in general are computationally powerful and very promising for tasks in which temporal information has to be processed. We’d like to remark that this is the case for virtually any task or application interacting with the real world. We have tested th...
متن کاملDynamic Memory via Delay Coincidence Detection for Robot Maze Navigation
This paper demonstrates that spatiotemporal patterns produced by spiking neural networks of coincidence detection cells can be effectively decoded. We begin by recounting a procedure to find appropriate decoding delays. We expand on previous work by showing that an accurate coincidence detection threshold for decoding cells can be determined from observed dynamics. We then show that randomly ge...
متن کاملA Neuromorphic CMOS Family and its Application
Silicon circuits that mimic the nervous systems of insects and other animals represent the future of neurocomputing. They can perform various neural functions because the microstructures of a nervous system are replicated on their silicon chips. Since recent functional models of spiking neural networks tend to use spiking neurons, neuromorphic engineers have developed CMOS neural systems with s...
متن کاملA comparison between spiking and differentiable recurrent neural networks on spoken digit recognition
In this paper we demonstrate that Long Short-Term Memory (LSTM) is a differentiable recurrent neural net (RNN) capable of robustly categorizing timewarped speech data. We measure its performance on a spoken digit identification task, where the data was spike-encoded in such a way that classifying the utterances became a difficult challenge in non-linear timewarping. We find that LSTM gives grea...
متن کاملLong short-term memory and Learning-to-learn in networks of spiking neurons
Networks of spiking neurons (SNNs) are frequently studied as models for networks of neurons in the brain, but also as paradigm for novel energy efficient computing hardware. In principle they are especially suitable for computations in the temporal domain, such as speech processing, because their computations are carried out via events in time and space. But so far they have been lacking the ca...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2003